1. Field of the Invention
The present invention relates to the processing of computer graphics, and in particular to a method of and an apparatus for carrying out anti-aliasing when processing computer graphics.
The present invention will be described with particular reference to the processing of three dimensional graphics, although as will be appreciated by those skilled in the art, it is equally applicable to the processing of two-dimensional graphics as well.
2. Discussion of Prior Art
One problem that is encountered when displaying computer generated images is that the displayed image is quantised into the discrete pixel locations of the display, e.g. monitor or printer, being used. This limits the resolution of the image that is displayed and can produce unwanted visual artifacts, for example, where the resolution of the output display device is not high enough to display smooth lines. These effects are commonly referred to as “aliasing”.
FIG. 1 illustrates such aliasing effects. The lefthand side of FIG. 1 shows the image to be drawn, and the righthand side shows the actual image that is displayed. As can be seen, the desired smooth curve of the white object in fact has a jagged appearance on the display. This is aliasing. (In FIG. 1, each square represents a pixel of the display, and the crosses represent the points at each (x, y) pixel location for which the colour value for that pixel location is determined (sampled). For example, the pixel A in FIG. 1 is drawn as all white, because the colour sampling point for that pixel location falls within the white object. It should be noted that in FIG. 1 only sample crosses on the pixels of interest are shown, although in practice all pixels will be sampled.)
All aliasing artifacts could be removed by using a sufficiently high resolution display. However, the resolution of electronic displays and printers is typically limited, and so many graphics processing systems use other techniques to try to remove or reduce the effects of aliasing. Such techniques are typically referred to as anti-aliasing techniques.
One known anti-aliasing technique is referred to as supersampling or oversampling. In such an arrangement, plural colour samples are taken for each pixel location of the display, and then those plural samples are combined into a single colour when the pixel is displayed. This has the effect of smoothing or averaging the colour values from the original image at the pixel location in question.
FIG. 2 illustrates the supersampling process. In the example shown in FIG. 2, four colour values (sample points) are determined for each pixel in the display. (Each such sample can effectively be viewed as a “sub-pixel”, with each pixel in the display being made up of four such sub-pixels.) The four colour value samples (sub-pixels) for a given pixel are then combined (downfiltered) such that the final colour that is used for the pixel in the display is an appropriate average (blend) of the colours of the four colour samples taken for the pixel.
This has the effect of smoothing the image that is displayed, and, e.g., reduces the prominence of aliasing artifacts by surrounding them with intermediate shades of colour. This can be seen in FIG. 2, where the pixel A now has two “white” samples and two “black” samples and so is set to 50% “white” in the displayed image. In this way, the pixels around the edges of the white object are blurred to produce a smoother edge, based on, e.g., how many samples are found to fall on each side of the edge.
Supersampling in effect processes the screen image at a much higher resolution than will actually be used for the display, and then scales and filters (down samples) the processed image to the final resolution before it is displayed. This has the effect of providing an improved image with reduced aliasing artifacts, but requires greater processing power and/or time, since the graphics processing system must in effect process as many pixels as there are samples (such that, e.g., for 4× supersampling (i.e. where 4 samples are taken for each pixel location), the processing requirements will be four times as great than if there was no supersampling).
Other anti-aliasing techniques have therefore been proposed that, while still providing some improvement in image quality, have less processing requirements than supersampling.
One common such technique is referred to as “multisampling”. In multisampling, plural samples are again taken for each pixel that will make up the final display, but rather than determining a separate colour value for each sample, a single colour value is determined and applied to all the samples for a pixel that are found to belong to the same object in the final image. In other words multisampling calculates a single colour value for a given pixel for a given object in the scene, which colour value is applied to (reused for) all samples (subpixels) for the pixel that are covered by that object (in contrast to supersampling where a separate colour value is determined for each sample).
Because only a single colour value is used for multiple samples for a given pixel, multisampling is less processing intensive than supersampling and therefore can allow faster processing and performance than supersampling. However, there is a reduction in the quality of the displayed image as compared to supersampling, since although objects' edges are still sampled at a higher resolution, colours are not.
As is known in the art, 3D graphics processing is normally carried out by first splitting the scene to be displayed into a number of similar basic components (so-called “primitives”) to allow the 3D graphics processing operations to be more easily carried out. These “primitives” are usually in the form of simple polygons, such as triangles, and are usually described by defining their vertices.
Once the scene to be displayed has been divided into a plurality of graphics primitives, the graphics primitives are usually then further divided, as is known in the art, into discrete graphical entities or elements, usually referred to as “fragments”, on which the actual graphics processing operations (such as rendering operations) are carried out. Each such graphics fragment will represent and correspond to a given position in the primitive and comprise, in effect, a set of data (such as colour and depth values) for the position in question.
Each graphics fragment (data element) usually corresponds to a single pixel (picture element) in the final display (since as the pixels are the singularities in the final picture to be displayed, there will usually be a one-to-one mapping between the “fragments” the 3D graphics processor operates on and the pixels in the display). However, it can be the case that there is not a direct correspondence between “fragments” and “pixels”, for example where particular forms of post-processing such as down-scaling are carried out on the rendered image prior to displaying the final image.
Thus two aspects of 3D graphics processing that are typically carried out are the “rasterising” of graphics “primitive” (or polygon) position data to graphics fragment position data (i.e. determining the (x, y) positions of the graphics fragments to be used to represent each primitive in the scene to be displayed), and then “rendering” the “rasterised” fragments (i.e. colouring, shading, etc., the fragments) for display on a display screen.
(In 3D graphics literature, the term “rasterisation” is sometimes used to mean both primitive conversion to fragments and rendering. However, herein “rasterisation” will be used to refer to converting primitive data to fragment addresses only.)
The rastering process basically involves determining for the sampling point (or the plural sampling points where supersampling or multisampling is being carried out) for each pixel in the display whether the primitive in question covers that sampling point (or sampling points). A fragment or fragments having the appropriate (x, y) position(s) are then generated for rendering the sampling point or points that are covered by the primitive.
The rendering process basically involves deriving the data necessary to display each fragment. Such data typically comprises red, green and blue (RGB) colour values for each fragment (which will basically determine the colour of the fragment on the display), and a so-called “Alpha” (transparency) value for each fragment.
As is known in the art, this data is typically derived by carrying out individual rendering processes (steps) on each fragment (i.e. the data for that fragment) one after another in a linear or pipelined fashion. Thus, for example, each fragment is firstly allocated initial RGB and alpha values, based on, e.g., the (x, y) position of the fragment, and the colour and transparency data recorded for the vertices of the primitive to which the fragment belongs. Operations such as texturing, fogging, and blending, etc., are then carried out on the fragment data in succession. These operations modify the initial RGB and alpha values set for each fragment, such that after the last processing operation, each fragment has an appropriate set of RGB and alpha values to allow that fragment to be displayed correctly on the display screen.
The final sets of RGB and alpha values to be used for display of the fragments are stored in a memory (commonly referred to as a frame buffer) as an array of fragment data that corresponds to the pixel array of the display. The data in the frame buffer is then used to set the pixels of the display when the image is to be displayed.
In many graphics rendering systems there will typically be an intermediate memory that stores the final fragment data and from which that data is transferred to the frame buffer for display. For example, in a deferred or tile-based rendering system, fragment data will first be stored in one or more tile buffers before being transferred to the frame buffer. Even in an immediate mode render there may be an intermediate buffer that receives and stores the rendered fragment data before it is transferred to the frame buffer.
Another important aspect of the rendering process (for 3D graphics rendering at least) is to determine whether a fragment for a given primitive will actually be seen when the final scene is displayed. This determination is typically carried out, as is known in the art, by using depth (Z) values that are set for each fragment in a primitive as it is rendered.
Thus as well as storing RGB and alpha values for each rendered fragment, a depth value is stored as well. Then, when a new fragment for a given (x, y) position is to be rendered, (e.g. because it belongs to a different primitive that covers the same sampling point), its depth value is compared with the depth value of the fragment currently stored for the relevant fragment position in the memory (e.g. frame or tile buffer) storing the final fragment values, to determine whether the new fragment will be seen instead of the currently stored fragment. If the new fragment will be seen, it is rendered and its final fragment data (colour value, etc.) is stored in place of the existing fragment data stored for the fragment position in question.
Thus at the end of the rendering process for a given scene to be displayed, fragment data comprising at least colour (and, e.g., depth) values will be stored across an array of fragment (x, y) positions, which data will then be used to colour the pixels of the display to display the scene.
In the simplest case, where there is a single sampling point for each pixel of the display, each stored fragment position will correspond to a single pixel having that position in the display (i.e. there will be a one-to-one mapping between the fragments and the pixels of the display), such that the fragment data stored for a fragment position will be used to set the pixel colour for the pixel having the corresponding position on the display device.
However, in the case of supersampling and multisampling, there will or may not be such a one-to-one mapping between the fragments and the pixels in the final display.
For example, in the case of supersampling, the rendering process will render and store a fragment with a complete set of fragment data (e.g. including at least colour and depth values in the case of 3D graphics processing) for each sample of the image that is taken. Thus, for example, for 4× supersampling, four fragments will be rendered for each pixel in the final display, and the fragment data for each of those fragments stored as separate fragment data. The fragment data for each set of four stored fragments will then be appropriately combined (downsampled) to give a data set (colour value) to be used for the relevant pixel of the display. Thus in such an arrangement, four sets of fragment data will be stored for each pixel in the final display.
In the case of multisampling, multiple samples are again taken for each pixel when the image is rasterised into fragments. However, all the samples for a given pixel that are covered by the primitive in question are rasterised to a single fragment (rather than as separate fragments as would be the case for supersampling). This single fragment is then rendered, thereby giving a single, common set of fragment data (e.g. depth value, colour value, etc.), for all the sampling points of the pixel covered by the primitive, which set of fragment data is then stored, as before.
While both supersampling and multisampling offer advantages in terms of reducing aliasing artifacts, the Applicants believe that there remains scope for improvement in known anti-aliasing techniques.